Crate kalosm_language_model
source ·Expand description
Language Model
This crate provides a unified interface for language models. It supports streaming text, sampling, and embedding.
Usage (with the RPhi implementation crate)
use rphi::prelude::*;
#[tokio::main]
async fn main() {
let mut model = Phi::default();
let prompt = "The capital of France is ";
let mut result = model.stream_text(prompt).await.unwrap();
print!("{prompt}");
while let Some(token) = result.next().await {
print!("{token}");
}
}
Re-exports
pub use kalosm_sample;
Structs
- An embedder that uses OpenAI’s API for the Ada embedding model.
- A builder for the Ada embedder.
- The embedding space for the Ada embedding model.
- A type-erased session.
- A vector space for the DollySevenBSpace model.
- An embedding represents something about the meaning of data. It can be used to compare the meaning of different pieces of data, cluster data, or as input to a machine learning model.
- A builder for the
ModelExt::generate_text
method. - Parameters to use when generating text.
- A model that uses OpenAI’s API.
- A builder for gpt-3.5-turbo
- A model that uses OpenAI’s API.
- A builder for text-davinci-003
- A vector space for the GuanacoSpace model.
- A vector space for the LargePythiaSpace model.
- A vector space for the LlamaSevenChatSpace model.
- A vector space for the LlamaThirteenChatSpace model.
- A vector space for the MptBaseSpace model.
- A vector space for the MptChatSpace model.
- A vector space for the MptInstructSpace model.
- A vector space for the MptStorySpace model.
- A vector space for the OrcaSpace model.
- A vector space for the StableLmSpace model.
- A builder for the
ModelExt::stream_text
method. - The result of a structured parser stream.
- A marker type for models that do not support synchronous generation.
- A vector space for the TinyPythiaSpace model.
- This is a wrapper around a tokenizer to ensure that tokens can be returned to the user in a streaming way rather than having to wait for the full decoding.
- An untyped vector space that is not associated with a model. This can be used to erase the vector type from an embedding.
- A vector space for the VicunaSpace model.
- A vector space for the WizardLmSpace model.
Enums
- The type of GPT-NeoX model to use.
- The type of Llama model to use.
- Feedback to give to the model when generating text.
- The type of model to use.
- The type of MPT model to use.
Traits
- A model that has a chat format.
- A model that can be created asynchronously.
- A model that can be used to embed text. This trait is generic over the vector space that the model uses to help keep track of what embeddings came from which model.
- A model that can be used to generate text with an associated tokenizer.
- An extension trait for models.
- A session for a model.
- An extension trait for
Stream
s that provides a variety of convenient combinator functions. - A raw interface for a model that can be used to generate text synchronously. This provides a very low level interface to a model’s session:
- An extension trait for sync models.
- The type of a vector space marks what model the vector space is from. You should only combine vector spaces that come from the same model.
Type Aliases
- A trait object for a sync model.
- A trait object for an embedder.
- A trait object for a model.